KN YUR RE

Ryan Transfiguracion

July 23, 2018

Outline

  • Background
  • Team Roles
  • The Problem
  • The Data
  • Exploratory Analysis
  • Machine Learning Models
  • Conclusion
  • The Future and Recommendations

Background

What is League of Legends?

  • A free-to-play, multiplayer online battle arena (MOBA) videogame
  • Developed and published by Riot Games
  • ~100 million active players a month
  • ~100k viewers during peak hours on Twitch.tv
  • Active and popular professional eSports scene
  • Numerous 3rd party websites and books on gameplay strategies

How is League of Legends Played?

  • Primary mode: Summoner’s Rift
  • 5 vs. 5 human players, plus AI minions for each team
  • Main objective: destroy enemy’s base (aka The Nexus)
  • Sub-objectives: destroy enemy’s defense turrents
  • Key objectives: slay dragons and Baron Nashor for boosts

How is League of Legends Played?

  • Players select 1 of 141 champions to represent them
  • Each champion has unique set of attributes and abilities
  • Gain power by accumulating: XP to level up, and gold to buy enhancement items
  • Accumulate XP/gold by killing minions, monsters, and enemy players, and taking objectives
  • Killed players must wait before respawn

Team Roles

What are Team Roles?

  • In standard team composition, each player is designated one of fives roles based on:
  • Their champion’s attributes and abilities
  • Their intended location on the map near start of a match (around 1:25-1:30) – called the laning phase
  • The Roles: Top Laner, Jungler, Mid Laner, Botlane Carry, Support

Top Laner

  • Fights in the top lane of the map during laning phase
  • Key attributes/responsibilities: front line tankiness, deal moderate-to-high damage, ability to safely initiate team fights

Jungler

  • Fights in the jungle areas of the map during laning phase
  • Key attributes/responsibilities: ability to efficiently kill most jungle monsters during laning phase, provide ganks for in-lane teammates, place wards on map
  • Can deal high burst damage to neutral monsters via Smite

Mid Laner

  • Fights in the middle lane of the map during laning phase
  • Key attribute/responsibility: deal high burst, ability power-based damage
  • Champion is usually a mage or an assassin

Botlane Carry

  • Fights in the bottom lane of the map during laning phase
  • Key attributes/responsibilities: back line squishiness, deal high sustained attack damage, be the highest damage dealer in match
  • Champion is usually ranged (e.g., guns, arrows, etc.)

Support

  • Fights/assists in the bottom lane of the map during laning phase
  • Key attributes/responsibilities: help Botlane Carry survive and achieve kills, provide healing, shielding, and/or other utilities during team fights, place wards on map

The Problem

The Problem

  • Humanly impossible to visually observe every match and record team roles manually.
  • Can we use match data and machine learning to distinguish all five team roles from each other?

Team Roles – Match Overview, Draft Pick Format

Assumed Team Roles – Draft Pick Match

Assumed Team Role Participant IDs – Draft Pick Match
TeamRole BlueTeam RedTeam
TOP 1 6
JUNGLE 2 7
MID 3 8
BOTCARRY 4 9
SUPPORT 5 10

Team Roles – Match Overview, Blind Pick Format

Team Roles – NA LCS 2018 Spring Split, All Matches

Predicted Roles (Rows) vs. Actual/Assumed Roles (Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
DUO, BOTTOM 77 0 57 77 0
DUO, MIDDLE 0 55 0 0 55
DUO_CARRY, BOTTOM 111 0 0 0 0
DUO_CARRY, MIDDLE 0 0 4 0 0
DUO_SUPPORT, BOTTOM 0 0 0 111 0
DUO_SUPPORT, MIDDLE 0 4 0 0 0
NONE, JUNGLE 9 173 58 8 57
SOLO, BOTTOM 5 0 0 38 0
SOLO, MIDDLE 0 2 115 0 0
SOLO, TOP 32 0 0 0 122

The Data

Data Overview

  • Data for all matches in the 2018 Spring Split seasons of the North America LoL Championship Series (NALCS), the Europe LoL Championship Series (EULCS), LoL Champions Korea (LCK), and LoL Master Series (LMS), plus the 2018 Mid-Season Invitational (MSI)
  • Each match data accessed using API domain https://acs.leagueoflegends.com/v1/stats/game/

Obtaining API and Match IDs

Sample Collection of Match IDs

Match ID Data, First Six Matches of NA LCS 2018 Spring Split
Region.ID Game.ID Hash.ID Blue.Team Red.Team
TRLH1 1002440062 a3b08c115923f00d Team Liquid Team Solo Mid
TRLH1 1002440076 c426d3d50426edb3 100 Thieves OpTic Gaming
TRLH1 1002440084 f0f86e52b6e472e9 Clutch Gaming Golden Guardians
TRLH1 1002440095 fd3b9331ff5312e3 Echo Fox FlyQuest
TRLH1 1002440106 d6441d4ec8f87534 Counter Logic Gaming Cloud9
TRLH1 1002440127 361bcc2e848641d2 OpTic Gaming Team Liquid

Cleaning Each Match for Player Stats

  • JSON response object contains two sub-objects (participants and participantIdentities) that contain indvidual player stats
  • participantIdentities contains 10 players’ usernames under sub-object player
  • participants contains two more sub-objects (stats and timeline)
  • basically, participants is flattened out, then joined with player
  • final product for single match is 10 rows and nearly 150 columns/variables
  • process repeated for all other matches, and tables are concatenated together

Synopsis of Match Player Datasets

  • NA LCS: 117 matches, 1170 observations
  • EU LCS: 115 matches, 1150 observations
  • LCK: 246 matches, 2460 observations
  • LMS: 144 matches, 1440 observations
  • MSI: 74 matches, 740 observations

Sample of MSI 2018 Match Player Dataset

summonerName kills deaths assists totalDamageTaken
265 EVS RonOP 0 0 14 16266
249 SUP Zeitnot 5 2 6 9628
393 TL Pobelter 0 4 1 10883
65 SUP SnowFlower 0 1 10 11097
497 RNG Mlxg 2 3 13 24331
463 RNG Xiaohu 3 3 7 16666
693 FW Maple 2 2 2 16058
508 FW Maple 7 1 5 9683
349 FNC Rekkles 0 2 4 11090
412 FW Moojin 0 2 2 24881
280 EVS RonOP 2 5 10 15238
480 EVS RonOP 0 3 4 10773

Creating Player Per-Match Average Datasets

  • Observations from the match player datasets are grouped together by team, player name, and then assumed team role
  • Each group’s variables’ statistics are calculated to mean (average) values.

Sample of MSI 2018 Player Per-Match Average Dataset

summonerName kills deaths assists totalDamageTaken
30 FNC Rekkles 3.2857143 1.500000 4.000000 12270.07
33 GMB Edward 0.8888889 2.666667 7.222222 14253.56
57 PGM YutoriMoyasi 2.5000000 2.500000 4.666667 10887.17
71 TL Joey 0.0000000 3.000000 1.000000 10864.00
9 SUP Stomaged 1.7000000 3.200000 7.300000 24117.40
22 FW Hanabi 1.5000000 2.166667 4.055556 21834.06
14 DW Shernfire 2.1666667 3.666667 4.166667 29793.67
11 DW Chippys 2.0000000 3.000000 2.666667 20034.83
13 DW k1ng 1.6666667 2.500000 3.666667 11871.17
70 TL Impact 1.7272727 2.272727 4.000000 23604.55
39 KBM Riyev 0.5000000 3.500000 11.833333 13495.00
40 KBM Titan 4.6666667 3.333333 6.333333 11865.33

Exploratory Analysis

Box Plots 1

Box Plots 2

Box Plots 3

Box Plots 4

Machine Learning Models

Feature Set for Machine Learning 1

variableName description
kills The number of enemy champions killed.
assists The number of enemy champions assisted in killing.
magicDamageDealt The amount of magic damage dealt.
physicalDamageDealt The amount of physical damage dealt.
magicDamageDealtToChampions The amount of magic damage dealt to enemy champions only.
physicalDamageDealtToChampions The amount of physical damage dealt to enemy champions only.
totalHeal The amount of health points the player has regained.

Feature Set for Machine Learning 2

variableName description
totalUnitsHealed The number of entities a player healed.
damageSelfMitigated The amount of health points that were not lost from damage.
totalDamageTaken The amount of damage a player took from various sources.
neutralMinionsKilled The number of neutral monsters killed by a player.
timeCCingOthers The weighted sum of all CC applied
totalTimeCrowdControlDealt The sum of all CC applied
champLevel The (experience) level of a player at the end of a match.

Feature Set for Machine Learning 3

variableName description
visionWardsBoughtInGame The number of wards (i.e. surveillance items) a player purchased.
wardsPlaced The number of wards a player placed in the arena.
wardsKilled The number of enemy wards a player destroyed.
physDmgToChampsToDmgTakenRatio Physical damage dealt to champions / Total damage taken
totalMinionsKilled The number of enemy minions killed.
damageTakenPerMinDeltas.0.10 The amount of damage taken per minute from 0:00 to 10:00 minutes.
creepsPerMinDeltas.0.10 The number of enemy minions killed per minute from 0:00 to 10:00 minutes.

Machine Learning Approach

  1. Put together all five datasets for player per-match averages, then split the entire set into two subsets: one for training and one for testing.
  2. Create/train a model using the k-means clustering algorithm on the training set. Since we already know how many team roles exist, we set k=5.
  3. Test/validate the model using the k-nearest-neighbor (knn) classification algorithm on the testing set. Since there are only five possible neighbors, and we want to classify each observation to exactly one group (team role), we set k=1.

Machine Learning Approach (continued)

  1. Put together all five datasets for single match player stats.
  2. Test the model using knn on the new dataset.
  3. Evaluate performance of Steps 2, 3, and 5 using a confusion matrix.

Note: features are scaled using z-score.

Training the Model

k-means: Creating Training Model (Predicted Rows vs. Actual Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
BOTCARRY 18 0 0 0 0
JUNGLE 0 22 0 0 0
MID 0 0 20 0 0
SUPPORT 0 0 0 17 0
TOP 0 0 0 0 19

Validating the Model

knn: Predict Team Roles w/Per-Game Averages Test Data Set (Predicted Rows vs. Actual Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
BOTCARRY 27 0 0 0 0
JUNGLE 0 31 0 0 0
MID 0 0 28 0 0
SUPPORT 0 0 0 24 0
TOP 0 0 0 0 27

Testing the Model (Individual Matches Dataset)

knn: Predict Team Roles Single Games Test Data Set (Predicted Rows vs. Actual Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
BOTCARRY 1321 1 10 0 68
JUNGLE 0 1378 1 14 4
MID 19 1 1321 0 120
SUPPORT 0 7 1 1376 0
TOP 52 5 59 2 1200

Confusion Matrix

## Confusion Matrix and Statistics
## 
##                           
## knn_pred_test_single_games BOTCARRY JUNGLE  MID SUPPORT  TOP
##                   BOTCARRY     1321      1   10       0   68
##                   JUNGLE          0   1378    1      14    4
##                   MID            19      1 1321       0  120
##                   SUPPORT         0      7    1    1376    0
##                   TOP            52      5   59       2 1200
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9477          
##                  95% CI : (0.9422, 0.9528)
##     No Information Rate : 0.2             
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.9346          
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: BOTCARRY Class: JUNGLE Class: MID
## Sensitivity                   0.9490        0.9899     0.9490
## Specificity                   0.9858        0.9966     0.9749
## Pos Pred Value                0.9436        0.9864     0.9042
## Neg Pred Value                0.9872        0.9975     0.9871
## Prevalence                    0.2000        0.2000     0.2000
## Detection Rate                0.1898        0.1980     0.1898
## Detection Prevalence          0.2011        0.2007     0.2099
## Balanced Accuracy             0.9674        0.9933     0.9619
##                      Class: SUPPORT Class: TOP
## Sensitivity                  0.9885     0.8621
## Specificity                  0.9986     0.9788
## Pos Pred Value               0.9942     0.9105
## Neg Pred Value               0.9971     0.9660
## Prevalence                   0.2000     0.2000
## Detection Rate               0.1977     0.1724
## Detection Prevalence         0.1989     0.1894
## Balanced Accuracy            0.9935     0.9204

Current Classification System

Predicted Roles (Rows) vs. Actual/Assumed Roles (Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
DUO, BOTTOM 538 0 395 538 0
DUO, MIDDLE 0 347 0 0 347
DUO, TOP 0 3 0 0 3
DUO_CARRY, BOTTOM 533 0 0 0 0
DUO_CARRY, MIDDLE 0 0 12 0 0
DUO_SUPPORT, BOTTOM 0 0 0 533 0
DUO_SUPPORT, MIDDLE 0 12 0 0 0
NONE, JUNGLE 91 985 402 66 397
SOLO, BOTTOM 15 0 0 255 0
SOLO, MIDDLE 0 23 583 0 22
SOLO, TOP 215 22 0 0 623

Conclusion

Summary

  • 21 out of ~150 variables were chosen for model’s feature set
  • k-means clustering on per-match averages training set was 100% accurate
  • Used centroids produced from clustering as the model for validation and testing sets
  • Model used for knn classification on per-match averages validation set was 100% accurate
  • Used same model on “real world” single match performances test set
  • knn classification on test set was 94.7% accurate

Caveats / Limitations

  • Sample size vs. population
  • Patches ==> possible meta changes
  • Normal / Blind Pick / vs. AI modes are lower stakes
  • Unique champions (e.g. Vladimir, Swain, Gangplank, Ivern)
  • Varying levels of competition

The Future and Recommendations

The Future and Recommendations

  • Larger datasets: virtually endless number of non-custom matches can be accessed off of one username alone due to recursive nature of accessing match history.
  • Additional data (i.e., the timeline API using match IDs)
  • Alternative k-means training
  • Creating models per division
  • Creating models per patch

Final Thoughts

  • Draft Pick game modes allow safe assumptions on team roles
  • Limits the number of matches analytics sites can use for analyzing champion and team role performances
  • While not absolutely perfect, model could provide excellent tool for classifying team roles in Blind Pick Summoner’s Rift game modes,where team roles cannot be assumed based on Participant ID alone.